Papers with span extraction head
What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing paradigms for multi-task training involve a shared pre-trained language model and a small, thin network (head) given an input, a target head is the head that is selected for outputting the final prediction. |
| Approach: | They examine the behaviour of non-target heads when given input that belongs to a different task than the one they were trained for. |
| Outcome: | The non-target heads exhibit emergent behaviour, which may explain the target task, or generalize beyond their original task. |